Filtering supplementary content using completion rate

ABSTRACT

Systems and methods record tracking events during a presentation of supplementary content items to determine whether viewer(s) stopped watching the supplementary content item or watched the supplementary content item in its entirety. This determination may provide value to content publishers or (multi video programming distributors (MVPDs)) through preventing such supplementary content items from showing, whether in certain contexts, to certain viewers, and/or to certain audience segments. In some instances, the systems and methods may utilize machine-learning model(s) to determine which combinations of supplementary content items, contextual signals, viewers, and/or audience segments are likely to abandonment a supplementary content item. Preventing these combinations from showing, improves viewer experiences, permits MVPDs to effectively utilize opportunities for presenting supplementary content to viewers, in order to maximize revenue yield paid by supplementary content providers (e.g., advertisers), and efficiently uses computing resources to present supplementary content items that viewers are unlikely to abandon.

CROSS-REFERENCE TO RELATED APPLICATION

This patent application is a continuation of and claims priority to U.S.Utility patent application Ser. No. 16/996,589, filed Aug. 18, 2020,which is fully incorporated herein by reference.

BACKGROUND

Content publishers, such as multichannel video programming distributors(MVPDs), derive income from the sale of advertising time to advertisersthat want to promote their goods and/or services. Oftentimes,advertisers intend to target those viewers that are likely to beinterested in their products and/or services. One common technique is totarget viewers according to a particular type of television programming,geographical area, and/or behavioral characteristics.

With certain advertisements, there is no easily discernable way todetermine its success or effectiveness. Additionally, someadvertisements may be repulsive, uninteresting, irrelevant, or otherwisefail to garner the attention of the viewer. In such instances, theviewer may change the channel, turn off his or her device, or otherwiseabandon the advertisement before it completes playback. Here, revenuesassociated with presenting future advertisement(s) are lost. Moreover,computing resources spent determining the advertisements to present areunrealized.

BRIEF DESCRIPTION OF THE DRAWINGS

The detailed description is set forth below with reference to theaccompanying figures. In the figures, the left-most digit(s) of areference number identifies the figure in which the reference numberfirst appears. The use of the same reference number in different figuresindicates similar or identical items. The systems depicted in theaccompanying figures are not to scale and components within the figuresmay be depicted not to scale with each other.

FIG. 1 illustrates an example environment for filtering supplementarycontent based at least in part on a probability of completion, accordingto an embodiment of the present disclosure. In some instances, anexchange (e.g., advertisement exchange and/or advertisement server) thatstores and/or presents supplementary content may determine probabilitiesassociated with viewers abandoning the supplementary content prior totheir completion. The exchange may represent a supplementary contentserver working in conjunction with a number of other supply-side and/ordemand-side systems. Based on the probabilities, the exchange may usethe probabilities to prevent supplementary content from being presentedto the viewer and/or audience segments under certain contexts.

FIG. 2 illustrates an example process for determining a probability of aviewer abandoning supplementary content and including the supplementarycontent on a block list, according to an embodiment of the presentdisclosure.

FIG. 3 illustrates an example process for associating contextualcharacteristic(s) with an abandonment or completion of supplementarycontent, according to an embodiment of the present disclosure.

FIG. 4 illustrates an example process for determining a completion rateof supplementary content for use in determining whether to present thesupplementary content, according to an embodiment of the presentdisclosure.

FIG. 5 illustrates an example process for determining supplementarycontent to present to a viewer, according to an embodiment of thepresent disclosure.

DETAILED DESCRIPTION

Described herein are, among other things, systems and methods forfiltering supplementary content based on a probability of completion. Insome instances, the systems and methods may receive tracking eventsduring a presentation of the supplementary content to determine whetherthe supplementary content was presented in its entirety. Thisdetermination may provide value to content publishers or MVPDs throughpreventing supplementary content from showing, whether in certaincontexts and/or to certain viewers of certain audience segments. Forexample, determining that the supplementary content was abandoned byviewers of an audience segment may subsequently prevent similarsupplementary content from being shown under/in similar contexts. Insome instances, the systems and methods may utilize machine-learningmodel(s) and/or approaches to determine which combinations ofsupplementary content, contextual signals, viewers, and/or audiencesegments are likely to result in an abandonment of the supplementarycontent. Preventing these combinations from showing may improve viewerexperiences, increase publisher revenue, increase viewer retention,efficiently use computing resources to present supplementary contentthat viewers are unlikely to abandon, and permit MVPDs to effectivelyutilize opportunities for presenting supplementary content to viewers.

MVPDs, such as such as Direct TV, Comcast, Charter, or Time Warner, orstreaming services, such as Hulu, fuboTV, or Sling provide content(e.g., film and television programs) to viewers. These services may beprovided to viewers through advanced television devices, such asover-the-top (OTT) devices that deliver content via the Internet toviewer devices. For example, the MVPDs may have applications running onviewer devices that present the content to the viewers. Duringpresentation of the content, content breaks are often inserted. Duringthese breaks, supplementary content (e.g., promotions, commercials,advertisements, etc.) is/are presented to the viewer. In some instances,the MVPDs interact or communicate with a supplementary content exchange(e.g., advertisement exchange, advertisement sever, Supply Side Platform(SSP) (e.g., SpotX), Demand Side Platform (DSP) (e.g., The TradeDesk),etc.) for obtaining supplementary content or supplementary content itemsfor presenting to the viewer. As used herein, advertisements,supplementary content, supplementary content item(s), etc. may be usedinterchangeably. However, in some instances, supplementary content mayencompass advertisements, additional content that is supplementary towhat the viewer is watching (e.g., info about the content,actors/actresses, directors, series, etc.), about to watch, downloaded,or material that is presented during a portion or in lieu of maincontent presented to the viewer.

The supplementary content exchange may determine supplementary contentfor presenting to the viewer. In some instances, the supplementarycontent may be selected based on one or more contextual signalsassociated with the viewer (e.g., viewer device type, language settings,screen resolution, time of day, etc.), content being presented to theviewer and/or characteristics associated with the content (e.g., genre,rating, identifiers, streaming content, live content, program, etc.),whether the viewer is viewing the content via an application on theviewer device or a through a website, one or more audience segment(s) towhich the viewer belongs, behavioral characteristics of the viewer,and/or a geographical area of the viewer. This information may be usedto select and/or filter supplementary content (e.g., advertisements)targeted towards the viewer and which the viewer is likely to watchand/or buy the presented goods and/or services. By way of example, ifthe viewer is watching content related to a golf, advertisers maypurchase space to present supplementary content for golf apparel, ratherthan another type of sports apparel, such as biking. Additionally,viewers in one geographical location may likely be more interested ingoods and/or services from a particular advertiser than viewers in adifferent geographical location. In such instances, the supplementarycontent exchange may receive this information from the MVPDs and/or theviewer devices, whether directly or indirectly. For example, the MVPDmay provide indications of the device type and/or the language settingsto the supplementary content exchange for use in selecting thesupplementary content. The supplementary content exchange may also storesuch contextual characteristics.

After selecting supplementary content, the supplementary contentexchange may transmit the supplementary content to the MVPD and theMVPDs cause the supplementary content to be presented to the viewer.Additionally, in some instances, more than one supplementary contentitem may be selected for a particular break for presenting to theviewer. Here, the MVPDs may cause the supplementary content to bepresented consecutively.

The MVPDs obtain revenue through showing the supplementary content. Forexample, in instances where the supplementary content is presented, theMVPDs obtain revenue from the advertisers through showing theirsupplementary content. However, in some instances, viewer(s) may abandonthe supplementary content (e.g., stop watching) and in such instances,this represents lost opportunities to present additional supplementarycontent to the viewer. Viewers may abandon the supplementary content byswitching to a different program, channel, turning off the viewerdevice, and/or switching to a different OTT MVPD. In such instances, theviewer may not watch an entirety of the supplementary content. Forexample, the supplementary content may be irrelevant, repetitive,uninteresting, too personal, and/or sensitive. In such instances, theMVPD may receive revenue associated with presenting that supplementarycontent but may lose opportunities to present subsequent supplementarycontent. For example, the MVPD may receive a packet or bundle ofadvertisements to present but only a subset of these advertisement maybe viewed by the viewer before abandonment.

To track a progress or completion of the supplementary content, thesupplementary content exchange may receive indications associated withthe supplementary content being displayed to the viewer. Theseindications (e.g., tracking events) may be received when thesupplementary content started playback, when each quartile (or portion)of the supplementary content completes playback, and/or when the entiresupplementary content completes playback. The indications may thereforebe received throughout the playback of the supplementary content for useby the supplementary content exchange to determine how much of thesupplementary content was played. For example, if the supplementarycontent exchange did not receive an indication that supplementarycontent completed playback, then the supplementary content exchange maydetermine that the viewer abandoned the supplementary content. In suchinstances, the supplementary content exchange may reference a lastreceived indication for knowing how much of the supplementary contentwas presented. Moreover, by failing to receive an indication that thesupplementary content completed playback, the supplementary contentexchange may infer that the viewer did not watch the entirety of thesupplementary content. Additionally, from the indications received, thesupplementary content exchange may determine a time or location withinthe supplementary content at which the viewer stopped watching.

By tracking a playback of the supplementary content, the supplementarycontent exchange may correlate the supplementary content with a negativeviewing experience or determine that the context in which thesupplementary content was presented resulted in viewer abandonment. Forexample, the supplementary content exchange may correlate viewer(s)abandoning the supplementary content with the context in which thesupplementary content was presented. From this, the supplementarycontent exchange may determine that viewers do not want to watch thesupplementary content in the context of the content being played.Additionally, the supplementary content exchange may determine whichcombinations of supplementary content, context, audience segments,and/or behavioral characteristics are likely to result in viewersabandoning the supplementary content and/or content in future instances.

Knowing information about the content being watched by the viewer,audience segments of the viewer, and/or behavioral characteristics ofthe viewer, the supplementary content exchange may determine thatcertain supplementary content should be blocked given a completion rateof the supplementary content. That is, the supplementary contentexchange may track a completion rate of the supplementary content andthis completion rate may be used when determining whether viewers arelikely to abandon the supplementary content. For example, if viewers ofan audience segment typically abandon supplementary content, undercertain contextual characteristic(s), the supplementary content may bewithheld from being presented to the viewer in such contexts.

In some instances, based on viewer abandonment and/or determining thatviewers are likely to abandon the supplementary content, thesupplementary content may be associated with certain contexts and/oraudience segments. These associations may be organized into a blocklist, where the block list represents a list of supplementary content(or a category or type of supplementary content) that are not to bepresented to viewers of certain audience segments in certain contexts.The supplementary content on the block list may have a high probability(e.g., above a threshold) that viewers of an audience segment will (orare likely to) abandon the supplementary content if presented. Theprobabilities may be determined before presenting the supplementarycontent and used when the supplementary content exchange determines thesupplementary content to present.

In some instances, the block lists may each be associated with acontext, audience segment, and/or behavioral characteristics. Forexample, viewers may enjoy watching particular supplementary content incertain contexts, but not another. Here, the particular supplementarycontent may be appropriate for presenting in the first context, but notthe latter. Additionally, or alternatively, regardless of the context,some supplementary content may not be relevant to the viewers and may beblocked entirely. These relationships may be determined by thesupplementary content exchange, in advance, for computing probabilitiesthat the viewers are likely or unlikely to abandon. Based on thegenerated probabilities, the supplementary content may be blocked orpermitted. Therein, the supplementary content exchange may use the blocklist(s) for determining supplementary content to present.

For example, the supplementary content exchange may receive a requestfrom the MVPD for supplementary content and the supplementary contentexchange may fulfill the request. The supplementary content exchange mayfulfill the request based on contextual signals (e.g., key value pairs,such as deviceType=FireTV) received from the MVPD, the generated blocklists, audience segments of the viewers, completion rates of thesupplementary content, and so forth. In some instances, thesupplementary content exchange may fulfill the request through acceptingbids from advertiser(s). After receiving the bids, the supplementarycontent exchange may rank the bid(s) based on one or more criteria(e.g., relevance, generated revenue, cost of supplementary content,etc.) for selecting one or more supplementary content to present to theviewer. The bid(s) may also be ranked according to the detailstransmitted in the supplementary content request (e.g., contextualsignals). Additionally, the supplementary content exchange may alsofilter supplementary content, or bid(s), that would otherwise violatethe block list(s) if presented. That is, using the previously generatedblock list(s) and/or determined probabilities, if one of the bidsincludes a supplementary content that is on a block list, thesupplementary content exchange may not consider that supplementarycontent for presentation to the viewer.

Once the supplementary content is/are determined, the supplementarycontent may be transmitted to the MVPD for presentation to the viewer.In some instances, the MVPD may stitch the supplementary content intothe content presented to the viewer and/or the MVPD may fetchsupplementary content as needed (e.g., during breaks). Additionally, insome instances, the MVPD may include supplementary content of its ownand may not communicate with the supplementary content exchange forreceiving supplementary content. Moreover, in some instances, thesupplementary content exchange may communicate directly with the viewerdevice for presenting the supplementary content.

During presentation of the supplementary content, and as introducedabove, the supplementary content exchange may track a viewing of thesupplementary content by the viewer. If, for example, the viewerabandons the supplementary content, the supplementary content exchangemay record this for use in updating the block list(s) for future use indetermining supplementary content that is/are likely to abandon. Forexample, the supplementary content exchange may employ themachine-learning model(s) to flag the supplementary content and avoidpresenting the supplementary content in future instances under similarcontext(s) and/or to similar viewers. In some instances, the blocklist(s) may be updated in real-time and/or during pre-schedule offlineupdates. In such instances, the supplementary content exchange mayaggregate feedback (e.g., abandoned or finished watching thesupplementary content) received from multiple viewers of an audiencesegment to update the block list.

Furthermore, the feedback received from the viewers permits themachine-learning model(s) to determine complex relationships between thecontent that the viewer is watching, the context in which the viewerwatches the supplementary content, the supplementary content itself,audience segment(s) to which the viewer belongs, and so forth. Forexample, in subsequent instances in which the MVPD submits supplementarycontent requests, the supplementary content exchange may utilize themachine-learning model(s) to identify supplementary content items toinclude on a block list based on the current content being viewed by theviewer, a completion rate of the supplementary content, and/or a contextin which the viewer is watching the content (e.g., device, time,location, language, etc.). As such, the supplementary content exchangemay use this criteria (e.g., contextual signals) to avoid returningsupplementary content items that are included on the block list or whichwould otherwise violate the block list(s). In some instances, themachine-learning model(s) may be trained for each audience segment andcontextual characteristic(s) combination to avoid pre-computing blocklist for specific viewers.

This refinement of supplementary content selection represents anefficient use of computing resources in the digital context. Forexample, in instances where supplementary content items are abandoned,the time and computing resources spent selecting these supplementarycontent items is unrealized. This represents an inefficient use ofcomputing resources that were expended to determine supplementarycontent for the viewer, but which were ultimately not viewed by theviewer. Maintaining block lists that includes combinations ofsupplementary content, viewers, and context that have a likelihood ofbeing abandoned permits the supplementary content exchange to returnsupplementary content that is/are likely to be watched in theirentirety. That is, knowing the probability that the viewer will abandonsupplementary content permits the supplementary content exchange fromdelivering those items for presentation to the viewer. Accordingly,presenting those supplementary content items that are unlikely to beabandoned represents an efficient use of computing resources and avoidsthe use of computing resources that would have been used to select andpresent supplementary content that subsequently would likely have beenabandoned. In such instances, the supplementary content exchange and/orthe MVPD more effectively utilize their computing resources to maximizeopportunities to present supplementary content items to improve viewerexperiences, increase publisher revenue, and/or increase viewerretention.

In some instances, the MVPD may utilize the supplementary contentexchange as a service for preventing the supplementary content on theblock lists to be blocked. For example, the supplementary contentexchange may offer services to the MVPD for filtering the supplementarycontent to increase viewer retention and/or viewing experiences whenengaging the MVPD.

The present disclosure provides an overall understanding of theprinciples of the structure, function, device, and system disclosedherein. One or more examples of the present disclosure are illustratedin the accompanying drawings. Those of ordinary skill in the art willunderstand and appreciate that the devices, the systems, and/or themethods specifically described herein and illustrated in theaccompanying drawings are non-limiting embodiments. The featuresillustrated or described in connection with one embodiment, or instance,may be combined with the features of other embodiments or instances.Such modifications and variations are intended to be included within thescope of the disclosure and appended claims.

FIG. 1 illustrates an example environment 100 for determiningadvertisement(s) 102 (e.g., supplementary content, supplementary contentitem, etc.) to present to viewer(s) 104 (which might be referred toherein singularly as “the viewer 104” or in the plural as “the viewers104”). As illustrated, the viewer(s) 104 may utilize viewer device(s)106 (e.g., e.g., mobile telephones, tablet devices, gaming consoles,televisions, laptop or desktop computers, etc., etc.) for streaming orwatching content 108 (e.g., television shows, programs, films, movies,audio, etc.). In some instances, the viewer device(s) 106 may be smartdevices themselves or include accessory devices that plug into orcommunicatively couple to the viewer device(s) 106, such as OTT devices.In this context, the OTT devices may plug directly into the viewerdevice(s) 106 for enabling applications and/or content streaming to theviewer device(s) 106.

The content 108 may be provided or supplied via a multichannel videoprogramming distributor (MVPD) 110. For example, the MVPD 110 may offerthe content 108 to the viewer(s) 104 for viewing, whether by asubscription, pay per view, trial, and so forth.

The MVPD 110 is shown including processor(s) 112 and memory 136, wherethe processor(s) 112 perform various functions and operations describedherein and the memory 136 may store instructions executable by theprocessor(s) 112 to perform the operations described herein. Forexample, the memory 136 is shown including or having access to theviewer(s) 104 and the content 108, for distributing or presenting to theviewer(s) 104.

The viewer(s) 104 may represent individuals who view the content 108provided by the MVPD 110. In some instances, the MVPD 110 may store thecontent 108 for providing to the viewer(s) 104 (via the viewer device(s)106) and/or the content 108 may be located on one or more additionalservers, databases, and/or third-party platforms. During presentation ofthe content 108, the MVPD 110 may insert the advertisement(s) 102 intothe content 108 or otherwise cause the advertisement(s) 102 to bepresented to the viewer(s) 104. In some instances, the content 108 mayinclude advertising breaks that instruct or signal the MVPD 110 tosupply the advertisement(s) 102 to the viewer(s) 104 and/or cause theadvertisement(s) 102 to be presented.

The MVPD 110 may communicatively couple to an advertisement exchange 114for obtaining the advertisements 102 for presenting to the viewer 104.For example, during an advertising break, the MVPD 110 may submit anadvertisement request 116 to the advertisement exchange 114. Theadvertisement request 116 may represent a request for advertisements 102for presenting to the viewer 104. In some instances, the advertisementexchange 114 is configured to utilize the advertisement request 116 forsearching and/or selecting advertisements 102 for presentation to theviewer 104. Generally, the advertisement exchange 114 may represent amarketplace of advertisement(s) 102 or a platform in whichadvertisement(s) 102 are exchanged. For example, the role of theadvertisement exchange 114 may be to return the advertisements 102, suchas an advertisement server, a SSP, and/or a DSP.

The advertisement exchange 114 is shown including processor(s) 118 andmemory 120, where the processor(s) 118 perform various functions andoperations described herein and the memory 120 may store instructionsexecutable by the processor(s) 118 to perform the operations describedherein. a bid request manager 122 of the advertisement exchange 114 maypost or request bids based on the advertisement request 116. In someinstances, the bids posted by the bid request manager 122 may indicatethe length of advertisement(s) 102 sought, information associated withthe viewer 104, and/or information associated with the viewer device 106(e.g., language) presenting the content 108. This information may beused to assist advertiser(s) 124 in determining whether they want tosubmit a bid for the advertisement break. That is, the advertiser(s)124, who generate and/or produce the advertisement(s) 102, may bid onadvertisement break(s) according to the specifics of the advertisementbreak.

The advertisement exchange 114 includes the advertisement selectionmanager 126 for managing the bids and ultimately selecting a bid. Forexample, upon receipt of the bids and the advertisement(s) 102, theadvertisement selection manager 126 may filter and/or rank the bidsand/or the advertisement(s) 102 associated with the bids. In someinstances, the advertisement selection manager 126 may filter the bidsbased on the criteria included in the advertisement request 116. In someinstances, the advertisement selection manager 114 may compile multipleadvertisement(s) 102 for presentation to the viewer 104. Regardless,upon the selection of the advertisement(s) 102, the advertisementexchange 114 may transmit the advertisement(s) 102, or data associatedwith the advertisement(s) 102, to the MVPD 110. In response, the MVPD110 may deliver the advertisement(s) 102 to the viewer device(s) 106 orotherwise cause the advertisement(s) 102 to be presented to the viewer104 in association with the content 108. The viewer device(s) 106presenting the content 108 may include applications (e.g., theapplications displaying the content 108) that integrate theadvertisement(s) 102 into the content 108. In some instances, the MVPD110 may stitch the advertisement(s) 102 into the content 108 and thenprovide the content 108 with the advertisement(s) 102 to the viewerdevice(s) 106.

The advertisement exchange 114 may have access to various information ofthe viewer(s) 104 for presenting advertisement(s) 102 or for use indetermining which advertisement(s) 102 to present, or to not present, tothe viewer(s) 104. For example, knowing the current content 108 beingpresented to the viewer(s) 104 may be used to determine relevant and/orirrelevant advertisement(s) 102. In some instances, this information maybe stored as contextual characteristic(s) 128 within the memory 120 ofthe advertisement exchange 114. The contextual characteristic(s) 128 mayinclude information associated with the content 108 being presented theviewer(s) 104 (e.g., program, show, channel, episode, movie, etc.), atime at which the content 108 is being presented, a current locationwithin the content 108 the viewer 104 is watching. The contextualcharacteristic(s) 128 may also relate to a geographical location of theviewer 104, a type of viewer device 106 used by the viewer 104 to watchthe content, a language setting of the viewer device 106, a screenresolution of the viewer device 106, and so forth. However, thecontextual characteristic(s) 128 may relate to, or include, otherinformation related to the context in which the viewer 104 is watchingthe content 108 (e.g., streaming versus live event, whether the content108 is accessed through an application or website) and/or attributes ofthe content 108 (e.g., genre, rating, etc.).

In some instances, the advertisement exchange 114 may receive thecontextual characteristics 128 via the MVPD 110 and ultimately, from theviewer device(s) 106. For example, the MVPD 110 may provide thecontextual characteristic(s) 128 to the advertisement exchange 114 aspart of the advertisement request 116. In some instances, the MVPD 110may store the contextual characteristic(s) 128 associated with theviewer 104 and/or the viewer device 106, and upon receiving anindication to present the advertisement(s) 102 during display of thecontent 108, may forward the contextual characteristic(s) 128 with theadvertisement request 116.

The advertisement exchange 114 may also utilize audience segment(s) 130of the viewer 104 for determining advertisement(s) 102 to present, or tonot present. In some instances, the audience segment(s) 130 maycorrespond to group(s) and/or classes that the viewer(s) 104 belong.Generally, the audience segment(s) 130 may divide or classify theviewer(s) 104 into distinct subgroups based on characteristics of theviewer(s) 104. Viewer(s) 104 within the same audience segment(s) 130 maywatch similar content 108 and/or have similar behaviors. For example, ifthe viewer 104 is a mom that lives in Seattle, Wash., the viewer 104 maybelong to an audience segment 130 corresponding to moms that live in ageographical area associated with Seattle, Wash. The audience segment(s)130 to which the viewer 104 is associated may additionally, oralternatively, be based on demographics or other viewer traits (e.g.,gender identity, age, ethnicity, income, lifestyle, education, purchasehistory, and so forth.) In some instances, the viewer(s) 104 may belongto multiple audience segments 130.

As shown, the memory 120 may store a completion rate 132 in associationwith the advertisement(s) 102. In some instances, each advertisement 102may have an associated completion rate 132 that represents a percentageor amount of time the advertisement 102 is viewed in its entirety. Insome instances, the advertisement exchange 114 may store multiplecompletion rates 132 for the advertisements 102. For example, thecompletion rate 132 may relate to an average completion rate for theadvertisement 102 across all contexts and across all audience segments130 (e.g., an overall completion rate). Additionally, the advertisementexchange 114 may store a completion rate that represents the completionrate for an audience segment 130 and each contextual characteristic 128.For example, a completion rate 132 may be stored for a certain audiencesegment 130 and for viewers 104 with a FireTV device.

In some instances, the advertisement(s) 102 may have any completion rate132, such as 98%, 98.5%, 99%, and so forth. In some instances, thecompletion rate 132 may correspond to a success metric of theadvertisement 102 and in instances where the completion rate 132 has acertain percentage (e.g., greater than a threshold), this may indicatethat the advertisement 102 is successful in capturing the attention ofthe viewer 104. That is, advertisement(s) 102 with high completion rates132 may be watched in their entirety by the viewer(s) 104.

During presentation of the advertisement(s) 102, the advertisementexchange 114 may determine the completion rate 132 via tracking aprogress of the viewer(s) 104. For example, the advertisement exchange114 may receive tracking event(s) 134 during a presentation of theadvertisement(s) 102 on the viewer device(s) 106. In some instances, theadvertisement exchange 114 may receive the tracking events 134 directlyfrom the viewer device(s) 106 or via the MVPD 110. Regardless, thetracking event(s) 134 may be used to determine a progress or completionof the viewer 104 watching the advertisement 102. For example, theadvertisement exchange 114 may receive the tracking events 134 when theadvertisement 102 started playing, when each quartile (or portion) ofthe advertisement 102 was played, and/or when an entirety of theadvertisement 102 was played. The tracking events 134 may therefore bereceived throughout the playback of the advertisement 102 for use by theadvertisement exchange 114 to determine how much of the advertisement102 was played. For example, if the advertisement exchange 114 did notreceive a tracking event 134 that the advertisement 102 completedplayback, then the advertisement exchange 114 may determine that theviewer 104 abandoned the advertisement 102. In such instances, theadvertisement exchange 114 may reference a last received tracking event134 for knowing how much of the advertisement 102 was presented.Moreover, by failing to receive the tracking event 134 that theadvertisement 102 completed playback, the advertisement exchange 114 mayinfer that the viewer 104 did not watch the entirety of theadvertisement 102. Additionally, from the tracking event(s) 134, theadvertisement exchange 114 may determine a time or location within theadvertisement 102 at which the viewer 104 stopped watching.

Based on the tracking event(s) 134 the advertisement exchange 114 mayupdate the completion rate 132 of that advertisement 102. Furthermore,the completion rate 132 of the audience segment(s) 130 to which theviewer 104 belongs may be correspondingly updated. In some instances,the completion rate 132 may be updated based on aggregated feedbackreceived from multiple viewers 104, over a certain period of time, andduring an offline workflow. For example, over a certain period of time,the advertisement 102 may have been shown to 1000 viewers 104 of acertain audience segment 130. For those 1000 viewer(s) 104, 900 of theviewers 104 may have completed the advertisement 102 and 100 viewers 104may have abandoned the advertisement 102. Correspondingly, thecompletion rate 132 for viewers 104 of audience segment 130 may be 90%.Additionally, the completion rate 132 for certain contextualcharacteristic(s) 128 may similarly updated based on the trackingevent(s) 134. Moreover, in instances where the viewer 104 belongs tomultiple (e.g., twenty) audience segments 130, the completion rate 132associated with these audience segment(s) 130 may be updated.

In some instances, the viewer 104 may abandon the advertisement to watchdifferent content 108, may turn off the viewer device 106, and/or mayswitch to watching the content 108 on another viewer device 106.However, knowing that the viewer(s) 104 stopped watching theadvertisement 102 provides insight into which advertisement(s) 102 topresent and/or to not present to the viewer 104 in future instancesand/or to like viewer(s) 104 in certain audience segments 130. For thelatter, knowing those advertisement(s) 102 to not present to the viewers104 in certain contexts may equate to saving time and computingresources spent filtering and presenting those advertisement(s) 102 thatthe viewer 104 is likely to abandon. Stated alternatively, identifyingadvertisement(s) 102 that the viewer 104 is likely to not watch preventscomputing resource(s) being spent identifying such advertisement(s) 102.That is, in instances where the advertisements 102 are abandoned, thetime and computing resources spent selecting these advertisements 102 isunrealized. This represents an inefficient use of computing resourcesthat were expended to determine the advertisements 102 for the viewer104, but which were ultimately not viewed by the viewer 104. Refrainingfrom presenting combinations of the advertisements 102, to the viewers104, and within a context in which the viewer 104 abandoned theadvertisement 102 permits the advertisement exchange to filter thoseadvertisements 102 that are likely to be watched in their entirety bythe viewer 104. Knowing a likelihood that the viewer 104 will abandonthe advertisements 102 permits the advertisement exchange 114 fromreturning such advertisements 102 for presentation to the viewer 104.Accordingly, requesting those advertisements 102 that are unlikely to beabandoned by the viewer 104 represents an efficient use of computingresources and avoids the use of computing resources that would have beenused to select/determine and present advertisements 102 thatsubsequently would have been abandoned. Additionally, knowing thecompletion rate 132 permits the advertiser(s) 124 to refineadvertisement campaigns.

In some instances, the advertisement exchange 114 may attempt toassociate the contextual characteristic(s) 128 with the advertisements102 that are not watched in their entirety or were otherwise abandoned.For example, if the viewer 104 did not watch the advertisement 102 inits entirety, the advertisement exchange 114 may draw correlationsbetween the contextual characteristic(s) 128, the viewer(s) 104, thecontent 108, the audience segment(s) 130, and/or the advertisement 102.If the viewer 104 belongs to a particular audience segment 130, andviewer(s) 104 of that audience segment 130 frequently abandon thatadvertisement 102, the advertisement exchange 114 may determine that theadvertisement 102 is not well suited for viewer(s) 104 of that audiencesegment 130. Moreover, the advertisement exchange 114 may determine thatviewer(s) 104 in the audience segment 130 typically abandon theadvertisement 102 during presentation of particular content 108 or withviewers having certain contextual characteristic(s) 128. In other words,the advertisement exchange 114 may correlate an abandonment of theadvertisement 102 with certain contextual characteristic(s) 128 of theviewers 104 in an audience segment 130.

To determine such correlations, the advertisement exchange 114 may, insome instances, use machine-learning (ML) model(s) 138 to determine thecorrelations or otherwise analyze the abandonment of the advertisements102. For example, the ML model(s) 138 may analyze the contextualcharacteristic(s) 128 associated with presenting the advertisement 102,the completion rate 132, the audience segment(s) 130, the content 108,or other information, for use in determining correlations associatedwith the advertisement 102, the content 108, and abandonments of theadvertisement 102. Such correlations may be used in future instances toavoid presenting certain advertisement(s) 102 to the viewer(s) 104 andbased on similar contextual characteristic(s) 128. Additionally, the MLmodel(s) 138 may be used before presenting a potential advertisement todetermine whether the advertisement 102 should be presented.

In some instances, the ML model(s) 138 may determine or generate aprobability 140 (which might be referred to herein singularly as “theprobability 140” or in the plural as “the probabilities 124”) for theadvertisement(s) 102 that represents a probability of the viewer 104abandoning the advertisement 102 (or, alternatively, completing theadvertisement 102 in its entirety). The individual probabilities 140 ofthe advertisements 102 may be determined by accessing data associatedwith the contextual characteristic(s) 128, the content 108, and soforth, of each viewer 104, providing the data as input to the MLmodel(s) 138, and generating, as output from the ML model(s) 138, theprobability 140 that is associated with the advertisement 102 beingabandoned. In some instances, the probability 140 may relate to alikelihood that the viewer 104 will not abandon the advertisement 102and/or will abandon the advertisement. Regardless, the probabilities 140output by the ML model(s) 138 may be machine-learned scores.

Machine-learning generally involves processing a set of examples (called“training data”) in order to train a machine-learning model(s). Amachine-learning model(s), once trained, is a learned mechanism that canreceive new data as input and estimate or predict a result as output.For example, a trained machine-learning model may comprise a classifierthat is tasked with classifying unknown inputs as one of multiple classlabels. In some cases, a trained machine-learning model is configured toimplement a multi-label classification task. Additionally, oralternatively, a trained machine-learning model may be trained to infera probability, or a set of probabilities, for a classification taskbased on unknown data received as input.

In the context of the present disclosure, the unknown input may becontextual characteristic(s) 128, the viewer(s) 104, the content 108,the audience segment(s) 130, and/or the advertisement 102, and the MLmodel(s) 138 may be tasked with outputting the probability 140 thatindicates, or otherwise relates to, a probability that the viewer 104will abandon the advertisement 102 (or view the entire advertisement102). This probability 140 may represent an expected completion rate 132of the viewer 104. If the probability 140 that is output by the MLmodel(s) 138 relates to a likelihood that the advertisement 102 will beabandoned by the viewer 104, the advertisement exchange 114 may placethe particular advertisement on block list(s) 142 of the viewer 104and/or viewer(s) of audience segment(s) 130 to which the viewer 104 isassociated with. In some instances, the advertisement 102 may be flaggedfor use in determining to not present the advertisement in certaincontext(s) and based on the contextual characteristic(s) 128. That is,in future instances, when determining whether to present a potentialadvertisement 102, the advertisement exchange 114 may compare thecontextual characteristic(s) 128 of a current viewer 104 with those ofcontextual characteristic(s) 128 stored in association with the audiencesegment 130 and which indicate viewer 104 abandoning the advertisement102. In such instances, the ML model(s) 138 may be trained for eachaudience segment 130 and contextual characteristic(s) 128 combination toavoid pre-computing block list(s) 142 for specific viewers 104.

The training data that is used to train ML model(s) 138 may includevarious types of data. In general, training data for machine-learningmay include two components, features and labels. However, in someinstances, the training data used to train the ML model(s) 138 may beunlabeled. Accordingly, the ML model(s) 138 may be trainable using anysuitable learning technique, such as supervised learning, unsupervisedlearning, semi-supervised learning, reinforcement learning, and so on.The features included in the training data may be represented by a setof features, such as in the form of an n-dimensional feature vector ofquantifiable information about an attribute of the training data. Thefollowing is a list of example features that can be included in thetraining data for training the ML model(s) 138 described herein.However, it is to be appreciated that the following list of features isnon-exhaustive, and features used in training may include additionalfeatures not described herein, and, in some cases, some, but not all, ofthe features listed herein. Example features included in the trainingdata may include, without limitation, a current program, show, episode,or content watched by the viewer 104, a time of day the viewer 104watches the content 108, a viewer device 106 used by the viewer 104 towatch the content 108, a language associated with the viewer 104 (and/orthe viewer device, the content 108, etc.), behavioral characteristics ofthe viewer 104 (e.g., shopping history, frequently watched shows,hobbies, interests, etc.), a screen resolution of the viewer device 106,demographics of the viewer 104, a geographical location of the viewer,content and/or specifics of the advertisement (e.g., advertiser,products, services, etc.), audience segments 130 of the viewer 104,completion rates 132 of the viewer 104, completion rates 132 ofadvertisement(s) 102 watched by audience segments 130 to which theviewer 104 is associated with, previously watched advertisements 102 ofthe viewer 104, and so forth.

To train the ML model(s) 138, the advertisement exchange 114 may storetraining advertisement(s) 148. The training advertisement(s) 148 may beinput to the ML model(s) 138 to determine if any of the trainingadvertisement(s) 148 have a probability 140 that exceeds the threshold.For example, the training advertisement(s) 148 may represent set ofadvertisements that have been shown to viewers 104 in audiencesegment(s) 130, and which are associated with certain contextualcharacteristic(s) 128. The advertisements within the trainingadvertisement(s) 148 would need to be shown enough times (e.g.,threshold) to the viewers 104 in order for the data set to be consideredsignificant enough to use the probability 140 produced by the ML model138. For example, if there are too few data points in the data set, thenthe data cannot be used for accurately predicting whether the viewer 104will likely abandon the advertisement 102. Take for example, that afirst viewer in an audience segment 130 abandons the advertisement andthat the completion rate 132 for that audience segment 130 for thatadvertisement 102 would be 0%. However, this does not necessarily meanthat the advertisement 102 should be placed on a block list 142. Inother words, a confidence of the probability 140 may be low given theamount of data used to train the ML model(s) 138. For each advertisement102, there may be an associated threshold at which the ML model(s) 138becomes usable, after training, and based on a sample size of thetraining data. Each ML model(s) 138 may have an associated confidence inthe output (e.g., the probability 140) for a specific set of contextualcharacteristic(s) 128. As such, enough data points may first becollected to ensure the reliability of the probability 140. If the datapoints for the advertisement(s) are not enough to exceed a threshold,then the advertisement 102 may not be considered for inclusion in theblock list(s) 142 and/or may not be refrained from being presented tothe viewer 104.

In some instances, as part of the training process, weights may beapplied to a set of features included in the training data, as derivedfrom the historical data. In some instances, the weights that are setduring the training process may apply to parameters that are internal tothe ML model(s) 138 (e.g., weights for neurons in a hidden-layer of aneural network). These internal parameters of the ML model(s) 138 may ormay not map one-to-one with individual input features of the set offeatures. The weights may indicate the influence that any given feature,parameter, or characteristic has on the probability 140 that is outputusing the ML model(s) 138.

The ML model(s) 138 may represent a single model or an ensemble ofbase-level machine-learning models, and may be implemented as any typeof machine-learning model. For example, suitable machine-learning modelsfor use with the techniques and systems described herein include,without limitation, neural networks, tree-based models, support vectormachines (SVMs), kernel methods, random forests, splines (e.g.,multivariate adaptive regression splines), hidden Markov model (HMMs),Kalman filters (or enhanced Kalman filters), Bayesian networks (orBayesian belief networks), expectation maximization, genetic algorithms,linear regression algorithms, nonlinear regression algorithms, logisticregression-based classification models, or an ensemble thereof. An“ensemble” can comprise a collection of machine-learning models whoseoutputs (predictions) are combined, such as by using weighted averagingor voting. The individual machine-learning models of an ensemble candiffer in their expertise, and the ensemble can operate as a committeeof individual machine-learning models that is collectively “smarter”than any individual machine-learning model of the ensemble.

Using the ML model(s) 138 may therefore identify complex relationshipsbetween the contextual characteristic(s) 128, the viewer(s) 104, thecontent 108, the audience segment(s) 130, and/or the advertisement 102.For example, the ML model(s) 138 may learn to associate certaincontextual characteristic(s) 128 of the viewer(s) 104 and/or theaudience segment(s) 130 to indicate that viewer(s) 104 are likely orunlikely to abandon advertisement(s) 102. By way of illustration, basedthe contextual characteristic(s) 128, the ML model(s) 138 may determinethat an audience segment 130 corresponding to forty-year old males, whoare watching sports on a mobile phone in the afternoon, are likely toabandon an advertisement 102 relating to cleaning products and/orservices. In future instances, when the advertisement exchange 114presents advertisements 102 to the viewer 104, and the viewer 104 is aforty-year old male, who is watching sports on a similar device and/ortime, the MVPD 110 may avoid causing advertisement 102 related tocleaning products and/or services being presented.

Introduced above, the ML model(s) 138 may identify, generate, ordetermine the block list(s) 142 that represent advertisements 102 thatare likely to be abandoned by certain viewer(s) 104, during certaininstance(s), and based on the contextual characteristic(s) 128, thecontent 108, the viewer(s) 104, and/or the audience segment(s) 130. Theblock list(s) 142 may be used for filtering or determiningadvertisement(s) 102 for the viewer(s) 104. For example, the ML model(s)138 may compute which combinations of advertisement(s) 102, viewer(s)104, audience segment(s), content 108, contextual characteristic(s) 128,and/or advertisement(s) 102 correlate to a probability 140 of the viewer104 abandoning the content 108 and/or the advertisement 102. In suchinstances, the ML model(s) 138 may block these combinations from beingpresented to the viewer 104, via the block list(s) 142. In someinstances, the ML model(s) 138 may utilize the completion rate 132 ofthe advertisement 102 for determining whether the viewer 104 is likelyto abandon the advertisement 102. However, as noted above, enoughfeedback (e.g., abandonment/completion) may first be collected to ensurethe reliability of the probability 140. If the feedback for theadvertisement(s) 102 are not enough to exceed a threshold, then theadvertisement 102 may not be considered for inclusion in the blocklist(s) 142. That is, despite one viewer 104 abandoning an advertisement102, the advertisement 102 may not be placed on a block list(s) 142 foran audience segment 130 until feedback is received from multiple (or apredetermined amount) of viewer(s) 104 within the audience segment 130.Stated alternatively, the advertisement(s) 102 would need to be shownenough times to the viewers 104 in order for the feedback to beconsidered significant enough to use the probability 140 produced by theML model 138.

The use of the ML model(s) 138 allows for accurately predicting whetherthe content 108 and/or the advertisement(s) 102 are likely to beabandoned, leading to increased viewer experiences and fewer instancesof presenting advertisement(s) 102 that are abandoned by the viewer(s)104. In some instances, the ML model(s) 138 may learn to predict whichadvertisement(s) 102 are likely to be abandoned by the viewer(s) 104and/or which advertisement(s) 102 are unlikely to be abandoned. In thismanner, noted above, advertisement(s) 102 with high probabilities (e.g.,above threshold) may likely be abandoned by the viewer 104 and may bepopulated within the block list(s) 142. Advertisement(s) 102 with lowprobabilities (e.g., below threshold) may not likely be abandoned by theviewer 104. In some instances, the advertisement exchange 114 may onlytrack or record those advertisement(s) 102 that are likely to beabandoned (e.g., via the block list(s) 142) and may not keep track ofthose advertisement(s) 102 in which the viewer 104 is likely to watch inits entirety. Although the use of a threshold is described as oneexample way of determining whether to include an advertisement 102 onthe block list(s) 142, other techniques are contemplated, such asclustering algorithms, or other statistical approaches that use scoresfor use in determining whether to block certain advertisement(s) 102from the viewer(s) 104, or a particular audience segment 130.

In some instances, the threshold utilized to determine whether theprobability is indicative of the viewer 104 abandoning the advertisement102 may be based relative to the completion rate 132 of otheradvertisements 102. For example, advertisement completion rates 132 onthe viewer device(s) 106 may be, on average, approximately 99.5%. If thedetermined probability 140 is 97%, for example, the advertisement 102may be prevented from being shown. That is, the 2.5% difference incompletion rate 132 may be used to restrict the advertisement 102 frombeing shown.

The ML model(s) 138 is/are retrainable with new data in order to adaptthe ML model(s) 138 to understand advertisement(s) 102 to withhold frompresenting to the viewer(s) 104. For example, as the contextualcharacteristic(s) 128, behavioral characteristics, content 108, and/orthe advertisement(s) 102 change, new correlations become available. TheML model(s) 138 may be retrained in instances where viewer(s) 104abandoned the advertisement(s) 102 and via information obtainedassociated with the abandonment of the advertisement 102 (e.g., a topicof the advertisement 102, current content 108 being present,geographical area, contextual characteristic(s) 128, and so forth). Assuch, the ML model(s) 138 may provide insight to not present certainadvertisement(s) 102 to limit the number of viewer(s) 104 who abandonthe content 108 and/or the advertisement(s) 102. In some instances, theML model(s) 138 may be updated during scheduled background workflowsand/or according to predetermined schedules (e.g., once per hour, onceper day, etc.).

As such, the advertisement exchange 114 may use the ML model(s) 138 todetermine which combinations of advertisement(s), audience segment(s)130, and/or contextual characteristic(s) 128 correlate to probabilities140 of the viewer 104 abandoning the content 108 and/or theadvertisement(s) 102. In either event, the abandonment of the viewer 104represents lost revenue associated with showing future advertisement(s)102. In an effort to overcome this deficiency, the results of the MLmodel(s) 138 may block those combinations from showing to increaseviewer retention.

The block list(s) 142 are used by the advertisement exchange 114 whenrequesting advertisements and when filtering the advertisements 102. Forexample, during advertising break(s) the MVPD 110 may communicate withthe advertisement exchange 114 for obtaining advertisement(s) 102presented to the viewer(s) 104. As part of this request, the MVPD 110may also transmit the contextual characteristic(s) 128, details of theviewer 104 and/or the viewer device(s) 106, and/or the content 108currently being played. The advertisement exchange 114 may then use theadvertisement request 116 for determining which advertisement(s) 102 topresent and/or to refrain from presenting.

In some instances, each of the advertisement(s) 102 may be associatedwith an advertisement identification that is used to uniquely identifyeach of the advertisement(s) 102. These advertisement identification(ID) may be stored on the block list(s) 142 and used when filteringadvertisement(s) 102 presented to the viewer 104. However, someadvertisement(s) 102 may have hidden or unknown identifiers. In suchinstances, the advertisement exchange 114 may attempt to unwrap theadvertisement 102 for use in determining specifics of the advertisement102 (e.g., the advertiser 124, content, topic, goods, services, etc.).

In some instances, the block list(s) 142 and/or the ML model(s) 138 mayprovide the advertiser(s) 124 with information for optimizingadvertisement campaigns. For example, the advertiser(s) may changetargeting an advertisement 102 to better target an audience that may bemore likely to purchase the product and/or services. In such instances,the advertiser(s) 124 may avoid spending resource(s) in certain contextsand/or on certain audience segment(s) 130 that are likely to not watchtheir advertisement(s) 102 in their entirety. Knowing this informationat the outset may avoid the MVPD 110, the advertisement exchange 114,and/or the advertiser(s) 124 expending computing resource(s) associatedwith the selection of advertisement(s) 102 that ultimately have a highprobability of being abandoned by the viewer 104. Such instances alsorepresent a loss of revenue for the MVPD 110. For example, in instanceswhere the viewer(s) 104 abandoned an advertisement, the MVPD 110 missesout on opportunities to collect revenues associated with furtheradvertisement(s) 102 selected for presenting to the viewer 104.

The viewer device(s) 106, MVPD 110, advertisement exchange 114, and/oradvertiser(s) 124 may communicate with one another via a network 144.For example, the MVPD 110 may send the advertisement request(s) 116 viathe network 144 to the advertisement exchange 114. The MVPD 110 may alsoprovide the content 108 and the advertisement(s) 102 to the viewerdevice(s) 106 via the network 144. The network 144 may represent anytype of communication network, including a data network, and may beimplemented using wired infrastructure (e.g., cable, CATS, fiber opticcable, etc.), a wireless infrastructure (e.g., RF, cellular, microwave,satellite, Bluetooth, etc.), and/or other connection protocols.

In some instances, the advertisement exchange 114 may represent anadvertisement server working in conjunction with a number of platforms,systems, and/or third-party databases. The advertisement exchange 114may communicate with these various platforms, systems, and/orthird-party databases via the network 144. For example, theadvertisement exchange 114 may work in conjunction with a number ofother supply-side advertising systems (e.g., SpotX) and/or demand-sideadvertising systems 150 (e.g., The TradeDesk) for obtaining theadvertisements 102, presenting the advertisements 102, and/or otherwisedetermining which of the advertisements 102 to present or to not presentto the viewers 104.

Although the above discussion relates to the advertisement exchange 114utilizing the ML model(s) 138 for generating block list(s) 142, in someinstances, the MVPD 110 may have access to or include the ML model(s)138. In such instances, the MVPD 110 may include information fortraining the ML model(s) 138 and/or generating the block list(s) 142.Moreover, in some instances, the MVPD 110, the advertisement exchange114, and/or the advertiser(s) 124 may be embodied within a singlesystem. Still, in some instances, the MVPD 110 may not communicate withthe advertisement exchange 114, but may have access to its ownadvertisement(s) 102 for presenting to the viewer(s) 104. Additionally,although FIG. 1 illustrates the MVPD 110 and/or the advertisementexchange 114 including certain components, the MVPD 110 and/or theadvertisement exchange 114 may include additional components, modules,engines, hardware, and/or software associated with determining theprobabilities 140 of the viewer(s) 104 abandoning the content 108 and/oradvertisement(s) 102.

As used herein, a processor, such as processor(s) 112 and/or theprocessor(s) 118 may include multiple processors and/or a processorhaving multiple cores. Further, the processor(s) may comprise one ormore cores of different types. For example, the processor(s) may includeapplication processor units, graphic processing units, and so forth. Inone implementation, the processor(s) may comprise a microcontrollerand/or a microprocessor. The processor(s) may include a graphicsprocessing unit (GPU), a microprocessor, a digital signal processor orother processing units or components known in the art. Alternatively, orin addition, the functionally described herein can be performed, atleast in part, by one or more hardware logic components. For example,and without limitation, illustrative types of hardware logic componentsthat may be used include field-programmable gate arrays (FPGAs),application-specific integrated circuits (ASICs), application-specificstandard products (ASSPs), system-on-a-chip systems (SOCs), complexprogrammable logic devices (CPLDs), etc. Additionally, each of theprocessor(s) may possess its own local memory, which also may storeprogram components, program data, and/or one or more operating systems.

The memory 136 and/or the memory 120 may include volatile andnonvolatile memory, removable and non-removable media implemented in anymethod or technology for storage of information, such ascomputer-readable instructions, data structures, program component, orother data. Such memory may include, but is not limited to, RAM, ROM,EEPROM, flash memory or other memory technology, CD-ROM, digitalversatile disks (DVD) or other optical storage, magnetic cassettes,magnetic tape, magnetic disk storage or other magnetic storage devices,RAID storage systems, or any other medium which can be used to store thedesired information and which can be accessed by a computing device. Thememory may be implemented as computer-readable storage media (“CRSM”),which may be any available physical media accessible by the processor(s)to execute instructions stored on the memory. In one basicimplementation, CRSM may include random access memory (“RAM”) and Flashmemory. In other implementations, CRSM may include, but is not limitedto, read-only memory (“ROM”), electrically erasable programmableread-only memory (“EEPROM”), or any other tangible medium which can beused to store the desired information and which can be accessed by theprocessor(s).

FIGS. 2-5 illustrate various processes related to determiningprobabilities associated with viewers abandoning advertisements andusing these probabilities for filtering advertisements. The processesdescribed herein are illustrated as collections of blocks in logicalflow diagrams, which represent a sequence of operations, some or all ofwhich may be implemented in hardware, software, or a combinationthereof. In the context of software, the blocks may representcomputer-executable instructions stored on one or more computer-readablemedia that, when executed by one or more processors, program theprocessors to perform the recited operations. Generally,computer-executable instructions include routines, programs, objects,components, data structures and the like that perform particularfunctions or implement particular data types. The order in which theblocks are described should not be construed as a limitation, unlessspecifically noted. Any number of the described blocks may be combinedin any order and/or in parallel to implement the process, or alternativeprocesses, and not all of the blocks need be executed. For discussionpurposes, the processes are described with reference to theenvironments, architectures, and systems described in the examplesherein, such as, for example those described with respect to FIG. 1,although the processes may be implemented in a wide variety of otherenvironments, architectures, and systems.

FIG. 2 illustrates an example process 200 for determining theprobability 140 of the viewer 104 abandoning advertisement(s) 102 andusing the probability 140 to refrain from presenting the advertisement102.

At 202, the process 200 may cause content to be presented to a viewer.For example, the MVPD 110 may cause content 108 (e.g., televisionstream) to be presented on the viewer device 106 associated with theviewer 104. In some instances, the viewer device 106 may includeapplications and/or interfaces of the MVPD 110 for presenting thecontent 108.

At 204, the process 200 may determine an audience segment associatedwith the viewer. For example, the advertisement exchange 114 maydetermine audience segment(s) 130 to which the viewer 104 is associatedwith. By way of example, if the viewer 104 is a female that lives inPortland, Oreg., and/or enjoys watching documentaries, the viewer 104may be associated with an audience segment 130 including these traits.Viewers 104 of the same audience segment 130 may have similar behaviors,interests, characteristic, and so forth. Discussed herein, the audiencesegment(s) 130 may be used when determining or selectingadvertisement(s) 102 for presenting to the viewer 104. In someinstances, as the viewer 104 may belong to multiple audience segment(s)130, the audience segment 130 selected at 204 may be based on thecurrent content 108 being displayed for targeting advertisements 102 tothe viewer 104.

At 206, the process 200 may determine one or more contextualcharacteristic(s) of the content, the viewer, and/or the viewer device.For example, based at least in part on the content 108 being presented,characteristic(s) of the viewer 104, and/or the type of viewer device106 (or characteristics associated therewith) the advertising exchange114 may determine contextual characteristic(s) 128. In some instances,the contextual characteristic(s) 128 may include a subject or topic ofthe content 108 being presenting, a channel or program of the content108, a position within the content 108 being viewed by the viewer 104, alocation/time at which the viewer 104 watches the content 108, a screenresolution of the viewer device 106, a type of viewer device 106, and soforth.

At 208, the process 200 may determine a first advertisement presented toviewers of the audience segment. For example, by determining theaudience segment 130 (or audience segments 130) to which the viewer 104belongs, the process 200 may determine a first advertisement that waspreviously presented to the viewers 104 of that audience segment 130. Insome instances, the first advertisement may be one of a plurality (e.g.,tens, hundreds, thousands, etc.) presented to viewer(s) of the audiencesegment 130.

At 210, the process 200 may determine, using one or moremachine-learning model(s), and based at least in part on the one or morecontextual characteristic(s), a probability that the viewer willcomplete watching the first advertisement. For example, in an effort toavoid the viewer 104 abandoning the content 108 and/or advertisement(s)102, the advertising exchange 114 may utilize the ML model(s) 138 todetermine a likelihood that the viewer 104 will finish watching theadvertisement 102. The likelihood, or the probability 140, mayrepresent, based on the one or more contextual characteristic(s) 128,whether the viewer 104 will abandon the first advertisement 102 and/orfinish watching the first advertisement 102. In some instances, the MLmodel(s) 138 may be previously trained from viewer(s) 104 of theaudience segment 130 who did not complete watching the firstadvertisement 102 and/or completed watching the first advertisement 102.As part of calculating the probability 140, the ML model(s) 138 maycompare the contextual characteristic(s) 128 of the viewer 104 withthose of the viewers within the audience segment 130. For example,certain contextual characteristic(s) 128 may indicate that viewers 104of the audience segment 130 do not abandon the first advertisement 102,while other contextual characteristic(s) 128 may indicate that viewers104 of the audience segment 130 abandon the first advertisement 102. Assuch, the contextual characteristic(s) 128 may be compared via the MLmodel(s) 138 to determine the probability 140 of the viewer abandoningthe first advertisement 102. Additionally, or alternatively, othercharacteristics may be used for determining the probability 140, such asdemographics of the viewer 104, a geographical location of the viewer104, purchasing histories of the viewer 104, and so forth. In suchinstances, the ML model(s) 138 may accept this information as input(s)for use in generating the probability 140.

At 212, the process 200 may determine whether the probability is greaterthan a threshold probability. For example, based on the calculatedprobability 140, the advertisement exchange 114 may compare theprobability 140 to the threshold probability. The threshold probabilitymay represent a threshold of the viewer watching the advertisement 102in its entirety or abandoning the advertisement 102. For example, theprobability 140 may be 98%. If the probability 140 is greater than thethreshold probability, this may indicate that the viewer 104 is notlikely to abandon the advertisement 102. However, if the probability 140is less than the threshold probability, this may indicate that theviewer 104 is likely to abandon watching the first advertisement 102.

If at 212, the process 200 determines that the probability is less thanthe threshold probability, the process 200 may follow the “NO” route andproceed to 214.

At 214, the process 200 may include the first advertisement on a blocklist associated with the viewer. For example, based at least in part ondetermining that the probability 140 does not satisfy the thresholdprobability, the advertisement exchange 114 may determine to not presentthe first advertisement 102 to the viewer 104 and accordingly, may placethe advertisement 102 on the block list 142. In other words, presentingthe first advertisement 102 based on the contextual characteristic(s)128, may lead to decreased viewer experiences and/or the viewer 104abandoning the first advertisement 102 and/or content 108.

From 214 the process 200 may loop to 208 to determine a secondadvertisement 102 presented to the viewers 104 of the audience segment130. After determining the second advertisement 102, the process 200 maycontinue to 210 and 212 for determining the probability 140. As such,the process 200 may repeat for determining whether to present theadvertisements 102 to the viewer 104. Moreover, in instances where theviewer 104 changes content 108, viewer device 106, and so forth, suchindications may be received for determining the probabilities 140 of theadvertisements 102 for another audience segment 130 and using thedifferent contextual characteristic(s) 128.

Alternatively, if at 212 the process 200 determines that the probabilityis greater than the threshold probability, the process 200 may followthe “YES” route and proceed to 216. At 212, the process 200 maydetermine to not include the first advertisement on the block listassociated with the viewer. For example, if the probability 140 does notindicate that the viewer 104 is likely to abandon the firstadvertisement 102, the process 200 may determine to present the firstadvertisement 102 (or that the first advertisement 102 is suitable forpresenting based on the contextual characteristic(s) 128). From 216, theprocess 200 may loop to 208 for determining an additional advertisement102 for determining the probability 140 of viewer abandonment (e.g.,second advertisement, third advertisement, etc.).

Accordingly, the process 200 and/or the methods herein may determineprobabilities 140 for a plurality of advertisement 102 and/or for aplurality of viewers 104 across audience segments 130 and based on thecontextual characteristic(s) 128. In such instances, the block list(s)142 of respective viewer(s) 104 and/or audience segments 130 may beupdated based on feedback of the viewer(s) 104 watching theadvertisement(s) 102 and indications whether the viewer(s) 104 watch theadvertisement(s) 102 in their entirety or abandoned the advertisement102. For example, the process 200 could repeat for a second audiencesegment 130 to which the viewer 104 belongs.

FIG. 3 illustrates an example process 300 for associatingadvertisement(s) 102 with contextual characteristic(s) 128 based onfeedback received from viewer(s) 104 viewing the advertisement(s) 102.

At 302, the process 300 may determine an advertisement to be presentedin association with content being present to a viewer. For example, theadvertisement exchange 114 may determine an advertisement 102 to bepresented in association with the content 108. The advertisementexchange 114 may determine the advertisement 102 in response toreceiving the advertisement request 116 from the MVPD 110. In someinstances, the advertisement exchange 114 may determine theadvertisement 102 based on the advertisement 102 not violating a blocklist of the viewer 104 and the completion rate 132 of the advertisement102 being greater than a threshold and/or the probability 140 ofcompletion being greater than the threshold.

At 304, the process 300 may determine one or more contextualcharacteristic(s) associated with the viewer. For example, theadvertisement exchange 114 may determine the contextualcharacteristic(s) 128, such as the program being presented or thecontent 108 to the first viewer 104, a viewer device 106 used by theviewer 104 to view the content 108, a time of day of the viewer 104watching the content 108, and so forth. Discussed herein, the contextualcharacteristic(s) 128 may be utilized by the advertisement exchange 114when gleaning information associated with the context in which theadvertisement 102 was presented, for use in determining whether theadvertisement 102 is to be presented to the viewer 104 and/or otherviewer(s) 104.

At 306, the process 300 may receive a first indication associated withthe viewer watching the advertisement, or that the advertisement isbeing presented to the viewer (e.g., being played back). For example,upon the advertisement 102 being presented on the viewer device 106, theadvertisement exchange 114 may receive the tracking event(s) 134 duringa playback of the advertisement 102. Initially, however, upon playback,the advertisement exchange 114 may receive an indication that theadvertisement is being played on the viewer device 106. In someinstances, and if the viewer 104 watches the advertisement 102, theadvertisement exchange 114 may receive the tracking event(s) 134throughout the presentation of the advertisement 102. Such indicationsmay be used to determine whether the viewer 104 completed watching theadvertisement 102 and/or whether the viewer 104 abandoned theadvertisement 102 (e.g., turned off viewer device 106, switchedprogram(s), etc.). In such instances, the advertising exchange 114 mayreceive multiple tracking event(s) 134 that represent a progress of theviewer 104 watching the advertisement 102. Such tracking event(s) 134may be used to determine an amount of the advertisement 102 watched bythe viewer 104.

At 308, the process 300 may determine whether a second indication isreceived associated with the viewer watching an entirety of theadvertisement. For example, based at least in part on receiving thetracking event(s) 134, the process 300 may determine whether the viewer104 watched an entirety of the advertisement 102. However, noted above,in some instances the advertisement exchange 114 may receive indicationsbetween the first indication and the second indication that represent aprogression of the viewer 104 watching the advertisement 102. Uponcompletion of the playback of the advertisement 102, the advertisementexchange 114 may receive an indication of such. That is, theadvertisement exchange 114 may receive an indication that theadvertisement 102 completed playback. Alternatively, if the process 300did not receive such an indication, the process 300 may determine thatthe viewer 104 did not complete watching the advertisement 102. That is,a lack of receiving an indication that the advertisement 102 completed aplayback may be used to determine that the advertisement 102 was notwatched in its entirety. In some instances, if the advertisementexchange 114 did not receive the second indication within a thresholdamount of time since receiving the first indication, the advertisementexchange 114 may infer that the viewer 104 abandoned the advertisement102 (e.g., timeout). If at 308, the process 300 determines that theviewer 104 did not watch the entirety of the advertisement 102, theprocess 300 may follow the “NO” route and proceed to 310.

At 310, the process 300 may associate the one or more contextualcharacteristic(s) with an abandonment of the advertisement. For example,as the viewer 104 did not complete watching the advertisement 102, theadvertisement exchange 114 may glean information and correlations toavoid presenting the advertisement(s) 102 to like viewer(s) 104 insimilar context(s) and under similar contextual characteristic(s) 128.In some instances, the advertisement exchange 114 may utilize the MLmodel(s) 138 for determining relationships between the contextualcharacteristic(s) 128, the content 108, the advertisement 102, thecompletion rate 132, and/or the audience segment 130. Theserelationship(s) may be used for determining whether to prevent theadvertisement 102 in future instances. For example, if the content 108was a science-fiction documentary, the first viewer 104 is a thirty-yearold male, and the advertisement 102 relates to kitchen appliances, theML model(s) 138 may determine that the advertisement 102 was potentiallyirrelevant to the viewer 104 and to not present the advertisement 102(given similar circumstances that the viewer 104 abandoned theadvertisement 102) to viewer(s) of the audience segment 130, undersimilar contextual characteristic(s) 128. As another example, if theviewer 104 abandoned the advertisement under certain contextualcharacteristic(s) 128 (e.g., first device, first location, first time),then under similar contextual characteristic(s) 128 (e.g., seconddevice, second location, second time, etc.) that are similar, viewers104 of the audience segment 130 may be likely to abandon theadvertisement 102.

In some instances, as part of associating the one or more contextualcharacteristic(s) 128 with an abandonment of the advertisement, theprocess 300 may include the advertisement 102 on a block list associatedwith the viewer. The block list 142 of the viewer 104 may includeadvertisements 102 that are not to be presented to the viewer 104 basedon certain contextual characteristic(s) 128 and/or under certainconditions (e.g., content, etc.).

At 312, the process 300 may determine a completion rate of theadvertisement. For example, the advertisement exchange 114 may determinean updated completion rate 132 of the advertisement 102. In someinstances, the completion rate 132 may represent viewers 104 viewing anentirety of the advertisement 102. For example, if the first indication(at 306) and the second indication (at 308) are received (i.e.,examining the advertisement 102 starting playback against theadvertisement 102 completing playback), this may indicate a completionof the advertisement 102 by the viewer 104. As such, based on the viewer104 not watching an entirety of the advertisement 102, the completionrate 132 of the advertisement 102 may be reduced. The completion rate132, may be used when determining whether to present the advertisement102 to other viewer(s) 104 that are included in similar audiencesegment(s) 130 as the viewer 104. In some instances, the advertisement102 may additionally, or alternatively, be placed on a block list 142 ofan audience segment 130 associated with the viewer.

Alternatively, if at 308, the process 300 determines that the indicationis indicative of the viewer watching an entirety of the advertisement,the process 300 may follow the “YES” route and proceed to 318.

At 318, the process 300 may associate the one or more contextualcharacteristic(s) with a completion of the advertisement. For example,based at least in part on the viewer watching an entirety of theadvertisement 102, the advertisement exchange 114 may determine that theadvertisement 102 is relevant, appropriate, or otherwise suited forpresentation during the content 108, to like viewers (e.g., the audiencesegment 130), and/or under similar contextual characteristic(s) 128. Aspart of this process and/or associating, the advertisement exchange 114may associate the advertisement with the contextual characteristic(s)128.

From 314, the process 300 may proceed to 312 whereby the completion rate132 of the advertisement 102 may be determined.

FIG. 4 illustrates an example process 400 for comparing contextualcharacteristic(s) 128 and determining whether to present advertisements102.

At 402, the process 400 may determine an advertisement presented toviewers of an audience segment. For example, the advertisement exchange114 may receive an indication or access a history of the viewers 104 ofthe audience segment 130 for determining advertisements 102 that werepresented to the viewers 104. The advertisement exchange 114 maymaintain a history of the advertisements 102 presented to the viewers inthe audience segment 130.

At 404, the process 400 may determine one or more contextualcharacteristic(s) associated with presenting the advertisement to afirst viewer of the audience segment. For example, the advertisementexchange 114 may determine a viewer device 106 used by the first viewer104 to watch content 108, a time/date in which the first viewer 104 iswatching the content 108, the type of content 108 being displayed, andso forth. However, the process 400 may determine any number ofcontextual characteristic(s) 128 of the viewer 104 and/or the content108 being displayed.

At 406, the process 400 may determine, among the viewers and based atleast in part on the one or more contextual characteristic(s), acompletion rate of the advertisement. For example, using the ML model(s)138, the advertisement exchange 114 may compare the one or morecontextual characteristic(s) 128 to locate viewer(s) 104 of the audiencesegment 130 that have similar contextual characteristic(s) 128. Forexample, the advertising exchange 114 may locate viewer(s) 104 that havea similar device type as the first viewer 104, viewer(s) 104 that werewatching similar content 108, and so forth. These contextualcharacteristic(s) 128 may represents inputs to the ML model(s) 138 fordetermining the completion rate 132. Based on locating the viewer(s) 104that have similar contextual characteristic(s) 128, the process 400 maydetermine a completion rate of those viewer(s) 104 watching theadvertisement 102. In doing so, the process 400 may determine thecompletion rate 132 of those viewer(s) completing the advertisement 102under similar contexts, conditions, and/or circumstances.

At 408, the process 400 may determine whether a confidence in thecompletion rate is greater than a threshold confidence. For example, theML model(s) 138 may have a certain confidence associated with thedetermined completion rate 132 or the probability 140 that the viewer104 will complete the advertisement 102. In some instances, theconfidence may be based at least in part on a sample size used to trainthe ML model(s) 138. For example, given that the ML model(s) 138 aretrained from historical data (e.g., the training advertisements 148) andthe contextual characteristic(s) 128, the output of the ML model(s) 138may have a corresponding confidence level. By way of example, based onone or more contextual characteristic(s) 128 determined at 404 (e.g.,type of viewer device 106, genre of content 108, live or streaming,etc.), the advertisement exchange 114 may determine how many otherviewers 104 of an audience segment 130 have similar contextualcharacteristic(s). This amount, or sample size, may represent or be usedto determine a confidence in the probability 140 that is output by theML model(s) 138. That is, if the ML model(s) 138 was trained on alimited set of data, the confidence may be lower than ML model(s) 138that was trained on a greater set of data. As such, based on thecontextual characteristic(s) 128, the ML model(s) 138 may have acorresponding confidence that the viewer 104 will watch theadvertisement 102, and using this confidence, the process 400 maydetermine the reliability of the output of the ML model(s) 138. A highconfidence may indicate that the probability 140 output by the MLmodel(s) 138 is trustworthy, while a lower confidence may indicate thatthe probability 140 output by the ML model(s) 138 is not trustworthy (ornot as trustworthy).

At 408, if the process 400 determines that the confidence is not greaterthan a threshold confidence, the process may follow the “NO” route andproceed to 410.

At 410, the process 400 may permit the advertisement to be displayed.For example, the advertisement exchange 114 may permit the advertisement102 to be presented in instances where the advertisement exchange 114 isnot confident in the completion rate 140 (e.g., low sample size for thecontextual characteristic(s) 128). That is, even if the process 400determines that the completion rate 140 is low, for example, the process400 may not be confident in such result. For example, if the ML model(s)138 was trained on a sample size of one (e.g., only one other viewer 104had the same or similar contextual characteristic(s) 128), theprobability 140 output by the ML model(s) 138 may not be trustworthyenough to refrain from presenting the advertisement 102.

Alternatively, if at 408 the process 400 determines that the confidencein the completion rate is greater than the threshold confidence, theprocess 400 may follow the “YES” route and proceed to 412. That is, ifthe ML model(s) 138 was trained on a sufficient sample size, forexample, the probability 140 output by the ML model(s) 138 may betrustworthy.

At 412, the process 400 may determine whether the completion rate isgreater than a threshold completion rate. For example, the completionrate 132 determined at 406 may be compared against a threshold. Thisthreshold may be determined statistically or historically acrosspreviously advertisements 102 being presented and may be defined by theadvertisement exchange 114 and/or the MVPD 110. If at 412 the process400 determines that the completion rate is not greater than thethreshold completion rate, the process 400 may follow the “NO” route andprocess to 414.

At 414, the process 400 may refrain from causing the advertisement to bepresented to the first viewer. For example, in instances where thecompletion rate 132 is less than the threshold completion rate, theadvertisement exchange 114 may refrain from causing the advertisement102 being displayed to avoid the first viewer 104 abandoning theadvertisement 102. That is, given the historical frequency of theviewer(s) 104 in the audience segment 130 abandoning the advertisement102, based on the contextual characteristic(s) 128, the probability 140of the viewer 104 completing the advertisement 102 may be less than thethreshold and as such, the advertisement 102 may be refrained forpresentation.

Alternatively, if at 412 the process 400 determines that the completionrate is greater than the threshold completion rate, the process mayfollow the “YES” route and proceed to 410 whereby the process 400 maypermit the advertisement to be played to the first viewer.

FIG. 5 illustrates an example process 500 associated with receiving arequest for an advertisement 102.

At 502, the process 500 may receive a request for an advertisement to bepresented to a viewer. For example, the advertisement exchange 114 mayreceive the advertisement request 116 from the MVPD 110. In someinstances, the advertisement request 116 may include the contextualcharacteristic(s) 128.

At 504, the process 500 may determine, among a plurality ofadvertisements, one or more first advertisements for presenting to theviewer. For example, the bid request manager 122 may receive bids fromthe advertiser(s) 124 associated with fulfilling the advertisementrequest 116. In some instances, the bid request manager 122 may postbids for the advertisements 102. Additionally, the bid request manager122 may not accept or receive bids for advertisements 102 that wouldotherwise violate the block list(s) 142 and/or have probabilities 140less than the threshold probability. That is, the advertisement exchange114 may utilize the previously generated block list(s) 142 for filteringout those advertisements 102 previously determined.

At 506, the process 500 may determine, among the one or more firstadvertisements, one or more second advertisements. For example, theadvertisement selection manager 126 may filter the advertisements 102submitted during the bidding process to remove advertisements that donot fulfill the request. In some instances, the advertisement selectionmanager 126 may remove advertisements 102 that were returned during thebidding process, but which are on block list(s) 142 of the viewer 104.In some instances, the advertisement selection manager 126 may unwrapthe advertisements 102 to determine content of the advertisement 102 inorder to determine whether or not to block the advertisement 102.Additionally, or alternatively, the advertisement selection manager 126may access metadata of the advertisement 102 to determine specifics ofthe advertisement 102.

At 508, the process may rank the one or more advertisements inaccordance with the request. For example, the advertisement selectionmanager 126 may rank the advertisements 102 that fulfill the request,according to specifics of the request. Here, the advertisement selectionmanager 126 may use knowledge about the audience segment 130, thecontent 108, the contextual characteristic(s) 128, etc. for determiningwhich advertisement(s) 102 the viewer 104 is likely to watch. As part ofthis process, the advertisement selection manager 126 may use one ormore machine-learning models for selecting a highest rankedadvertisement 102. For example, if the viewer 104 is watching a romancesitcom, the advertisement selection manager 126 may select anadvertisement about jewelry, beauty products, and/or clothes, ascompared to home improvement advertisements.

At 510, the process 500 may send at least one of the one or more secondadvertisements for viewing by the viewer. For example, after theadvertisement selection manager 126 selects an advertisement, theadvertisement exchange 114 may send this advertisement 102 to the MVPD110. Therein, the MVPD 110 may cause this advertisement to be presentedto the viewer 104.

While various examples and embodiments are described individuallyherein, the examples and embodiments may be combined, rearranged, andmodified to arrive at other variations within the scope of thisdisclosure.

Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific features or acts described. Rather,the specific features and acts are disclosed as illustrative forms ofimplementing the claims.

What is claimed is:
 1. A system comprising: one or more processors; andone or more computer-readable media storing instructions that, whenexecuted, cause the one or more processors to perform operationscomprising: receiving a request to present one or more supplementarycontent items during presentation of a content item to a viewer;identifying a potential supplementary content item to present to theviewer, the potential supplementary content item being associated with acompletion rate that represents viewers watching an entirety of thepotential supplementary content item; determining, for the potentialsupplementary content item, a probability associated with the viewerwatching the entirety of the potential supplementary content item, theprobability being based at least in part on the completion rate;determining that the probability is equal to or greater than a thresholdprobability; and based at least in part on the probability being equalto or greater than the threshold probability, causing the potentialsupplementary content item to be presented to the viewer.
 2. The systemof claim 1, the operations further comprising determining one or morecontextual characteristics associated with the content item, the one ormore contextual characteristics including at least one of: subjectmatter of the content item; a time of day at which the viewer views thecontent item; a language associated with a viewer device used to watchthe content item; or a type of the viewer device on which the contentitem is presented, wherein determining the probability is based at leastin part on the one or more contextual characteristics.
 3. The system ofclaim 1, the operations further comprising: determining a confidenceassociated with the probability; determining that the confidence isgreater than or equal to a threshold confidence; and wherein causing thepotential supplementary content item to be presented to the viewer isfurther based at least in part on the confidence being greater than orequal to the threshold probability.
 4. The system of claim 1, where thepotential supplementary content item comprises a first potentialsupplementary content item and the probability comprises a firstprobability, the operations further comprising: identifying a secondpotential supplementary content item to present to the viewer;determining, for the second potential supplementary content item, asecond probability associated with the viewer watching less than asecond entirety of the second potential supplementary content item;determining that the second probability is less than the thresholdprobability; and based at least in part on the second probability beingless than the threshold probability, refraining from causing the secondpotential supplementary content item to be presented to the viewer. 5.The system of claim 1, the operations further comprising: determiningthat the viewer watched less than the entirety of the potentialsupplementary content item; and associating contextual characteristicsof the potential supplementary content item with the viewer watchingless than the entirety of the potential supplementary content item,wherein associating the contextual characteristics with the viewerwatching less than the entirety of the potential supplementary contentitem is used to refrain from presenting the potential supplementarycontent item to one or more additional viewers having at least onecontextual characteristic of the contextual characteristics.
 6. Thesystem of claim 1, the operations further comprising: determining atleast one of: a subject matter of the content item presented to theviewer; a supplementary content item previously presented to the viewer;behavioral characteristics of the viewer; demographics of the viewer; ageographical location of the viewer; a type of a device used by theviewer to watch the content item; a screen resolution of the device; ora language setting of the device and providing at least one of thesubject matter, the supplementary content item, the behavioralcharacteristics, the demographics, the geographical location, the typeof device, the screen resolution, or the language setting as inputs to amachine-learning model, wherein the machine-learning model is configuredto determine the probability.
 7. A method comprising: determining one ormore characteristics associated with a content item being presented to aviewer; determining an audience segment associated with the viewer,wherein viewers within the audience segment have one or more behaviorcharacteristics that are in common with the viewer; determining asupplementary content item to present to the viewer; determining, basedat least in part on the one or more characteristics and the audiencesegment, a probability associated with the viewer watching an entiretyof the supplementary content item; determining that the probabilitysatisfies a threshold probability; and causing, based at least in parton the probability satisfying the threshold probability, thesupplementary content item to be presented to the viewer.
 8. The methodof claim 7, further comprising: associating the one or morecharacteristics, the supplementary content item, the audience segment,and the content item; and causing the supplementary content item to bepresented to one or more additional viewers based at least in part onthe associating.
 9. The method of claim 7, further comprisingdetermining a completion rate of the supplementary content item, thecompletion rate representing a percentage of viewers that watched theentirety of the supplementary content item, and wherein determining theprobability is further based at least in part on the completion rate.10. The method of claim 7, wherein the supplementary content itemcomprises a first supplementary content item, further comprising:identifying a second supplementary content item to present to theviewer; determining, for the second supplementary content item, a secondprobability associated with the viewer watching a second entirety of thesecond supplementary content item; determining that the secondprobability fails to satisfy the threshold probability; and based atleast in part on the second probability failing to satisfy the thresholdprobability, refraining from causing the second supplementary contentitem to be presented to the viewer.
 11. The method of claim 7, whereinthe viewer comprises a first viewer, further comprising: determiningthat the first viewer watched less than the entirety of thesupplementary content item; and refraining from presenting thesupplementary content item to a second viewer associated with the firstviewer.
 12. The method of claim 7, wherein the one or morecharacteristics comprise one or more first characteristics, furthercomprising: determining, among the viewers in the audience segment, oneor more second characteristics associated with the viewers watching theentirety of the supplementary content item; and determining that atleast one of the one or more first characteristics is a same as the oneor more second characteristics, wherein determining the probability isfurther based at least in part on determining that at least one of theone or more characteristics is the same as the one or more secondcharacteristics.
 13. The method of claim 7, wherein the viewer comprisesa first viewer, further comprising: receiving an indication associatedwith the first viewer watching the entirety of the supplementary contentitem; determining a second viewer associated with the first viewer; anddetermining, based at least in part on receiving the indication, topresent the supplementary content item to the second viewer.
 14. Themethod of claim 7, further comprising: determining a confidenceassociated with the probability; and determining that the confidence isgreater than or equal to a threshold confidence, wherein causing thesupplementary content item to be presented to the viewer is furtherbased at least in part on the confidence being greater than or equal tothe threshold probability.
 15. One or more non-transitory computerreadable media storing instructions executable by a processor, whereinthe instructions, when executed, cause the processor to perform actscomprising: determining a supplementary content item to present to aviewer; determining, for the supplementary content item, a completionrate that represents viewers associated with the viewer watching anentirety of the supplementary content item; determining, based at leastin part on the completion rate, a probability associated with the viewerwatching the entirety of the supplementary content item; determiningthat the probability is greater than or equal to a thresholdprobability; and causing the supplementary content item to be presentedto the viewer based at least in part on the probability being greaterthan or equal to the threshold probability.
 16. The one or morenon-transitory computer readable media of claim 15, wherein thesupplementary content item comprises a first supplementary content itemand the probability comprises a first probability, the acts furthercomprising: determining a second supplementary content item to presentto the viewer; determining a second probability associated with theviewer watching a second entirety of the second supplementary contentitem determining that the second probability is less than the thresholdprobability; and based at least in part on the second probability beingless than the threshold probability, refraining from causing the secondsupplementary content item to be presented to the viewer.
 17. The one ormore non-transitory computer readable media of claim 15, the actsfurther comprising: receiving an indication associated with the viewerwatching the entirety of the supplementary content item; and determiningto present the supplementary content item to one or more additionalviewers associated with the viewer.
 18. The one or more non-transitorycomputer readable media of claim 17, wherein the probability comprises afirst probability, the acts further comprising: determining, at a firstinstance in time, a first confidence associated with a secondprobability of the one or more additional viewers watching the entiretyof the supplementary content item; and determining, at a second instancein time and based at least in part on receiving the indication, a secondconfidence associated with the second probability of the one or moreadditional viewers watching the entirety of the supplementary contentitem, wherein determining to present the supplementary content item tothe one or more additional viewers associated with the viewer is basedat least in part on the second confidence.
 19. The one or morenon-transitory computer readable media of claim 15, the acts furthercomprising: receiving an indication associated with the viewer watchingless than the entirety of the supplementary content item; determining anupdated probability associated with one or more additional viewerswatching the entirety of the supplementary content item; and refrainingfrom causing the supplementary content item to be presented to the oneor more additional viewers.
 20. The one or more non-transitory computerreadable media of claim 15, the acts further comprising determining anaudience segment associated with the viewer, wherein viewers within theaudience segment have one or more behavior characteristics that are incommon with the viewer, wherein determining the supplementary contentitem is further based at least in part on the audience segment.